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In  opdracht  van  de  Koninklijke  Marine  werd  een  verkennende  studie  uitgevoerd  naar  de 
mens-machine  interface  voor  bet  besturen  van  “Maritime  Unmanned  Air  Vehicles”  (MUAV’s).  Dit 
rapport  beschrijft  een  simulatorexperiment  waarin  werd  nagegaan  hoe  effectief  operators  onder 
verschillende  verbaal/cognitieve  werklast  condities  een  bewegend  doel  kunnen  volgen  tijdens  een 
multi-MUAV  supervisietaak. 

De  experimentele  resultaten  laten  zien  dat  verhoogde  werklast  geen  invloed  had  op  de  mate  waarin 
het  doel  in  beeld  werd  gehouden.  Dit  was  gemiddeld  ongeveer  100%  bij  bijwerkfrequenties  van 
het  sensorbeeld  van  meer  dan  4  Hz,  en  daalde  naar  ongeveer  60%  bij  een  bijwerkfrequentie  van  1 
Hz.  Aangezien  de  proelpersonen  de  opdracht  kregen  om  primair  aandacht  ter  besteden  aan  de 
verbaal/cognitieve  taak,  kon  deze  verhoogde  aandacht  evenwel  niet  worden  gegeven,  vooral  niet 
bij  hoge  werklast.  Dit  leidde  tot  vergroting  van  de  volgfout. 

Mede  op  basis  van  eerder  onderzoek  (Van  Breda  &  Passenier,  1993)  wordt  geconcludeerd  dat  een 
hoge  bijwerkfrequentie  van  het  sensorbeeld  essentieel  is  voor  het  nauwkeurig  besturen  van 
MUAV’s.  Hoewel  er  momenteel  veel  tijd  wordt  gestoken  in  het  opvoeren  van  de  transmissie 
bandbreedte  van  de  downlink  zijn  er  slechts  beperkte  vorderingen  (Schwartz  e.a.,  1992;  NATO, 
1993;  Rochus  &  Garcia,  1995;  Van  Breda,  1995).  Vandaar  dat  wordt  aanbevolen  in  een  vervolg- 
studie  na  te  gaan  hoe  prestatie  en  orientatie  van  de  operator  kunnen  worden  verbeterd  bij  lage 
bijwerkfrequenties,  zodat  de  bandbreedte  van  de  downlink  voor  wat  betreft  de  besturing  van 
MUAV’s  minder  van  belang  is.  Dit  kan  onder  andere  door  integratie  van  synthetische  informatie 
over  MUAV  status  en  orientatie  in  het  sensorbeeld  (“synthetic  image  augmentation”).  Kunstmatige 
voertuigreferenties,  bijvoorbeeld,  of  een  synthetisch  perspectivisch  landschap,  gegenereerd  vanuit 
de  actuele  MUAV-positie  en  getoond  met  een  hoge  bijwerkfrequentie  (60  Hz),  kunnen  het  orien¬ 
tatie-  en  anticipatievermogen  verbeteren. 
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SUMMARY 

Under  contract  by  the  Royal  Netherlands  Navy,  an  exploratory  study  was  conducted 
concerning  the  man-machine  interface  and  task  characteristics  for  controlling  Maritime 
Unmanned  Air  Vehicles  (MUAVs).  This  report  describes  a  simulator  experiment  investigat¬ 
ing  how  effective  operators  can  track  a  moving  target  in  a  multi  MUAV  supervisory  control 
task,  under  different  verbal/cognitive  workload  conditions. 

Results  of  the  experiment  show  that  moderate  and  even  high  workload  conditions  did  not 
affect  the  mean  target  coverage  of  the  sensor  image.  This  was  about  100%  at  image  update 
frequencies  of  4  Hz  and  more,  decreasing  to  about  60%  at  1  Hz  update  frequency.  This  may 
be  explained  by  the  fact  that,  in  the  latter  case,  the  tracking  task  became  more  discontinu¬ 
ous,  which  demanded  more  attention  for  anticipation.  Since  the  subjects  were  instructed  to 
give  primary  attention  to  the  verbal/cognitive  task,  this  extra  attention  was  not  fully 
available,  in  particular  not  during  high  workload  conditions.  This  resulted  in  an  increase  of 
the  mean  viewing  (tracking)  error.  It  is  suggested  to  focus  further  research  on  ways  to 
improve  operator  performance  and  awareness  at  low  image  update  rates,  for  instance  by 
integrating  synthetic  information  on  orientation  and  MUAV  status  into  the  sensor  image 
(“synthetic  image  augmentation”). 
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Taakprestatie  bij  multi  “Maritime  Unmanned  Air  Vehicle”  besturing 
L.  van  Breda 

SAMENVATTING 

In  opdracht  van  de  Koninklijke  Marine  werd  een  verkennende  studie  uitgevoerd  naar  de 
mens-machine  interface  voor  het  besturen  van  “Maritime  Unmanned  Air  Vehicles” 
(MUAV’s).  Dit  rapport  beschrijft  een  simulatorexperiment  waarin  werd  nagegaan  hoe 
effectief  operators  onder  verschillende  verbaal/cognitieve  werklast  condities  een  bewegend 
doel  kunnen  volgen  tijdens  een  multi-MUAV  supervisietaak. 

De  experimentele  resultaten  laten  zien  dat  verhoogde  werklast  geen  invloed  had  op  de  mate 
waarin  het  doel  in  beeld  werd  gehouden.  Dit  was  gemiddeld  ongeveer  100%  bij  bijwerk- 
frequenties  van  het  sensorbeeld  van  meer  dan  4  Hz,  en  daalde  naar  ongeveer  60%  bij  een 
bijwerkfrequentie  van  1  Hz.  Dit  kan  worden  verklaard  doordat  in  de  laatstgenoemde  geval 
de  volgtaak  discontinu  werd,  waardoor  anticipatie  werd  bemoeilijkt  en  meer  aandacht 
vergde.  Aangezien  de  proefpersonen  de  opdracht  kregen  om  primair  aandacht  ter  besteden 
aan  de  verbaal/cognitieve  taak,  kon  deze  verhoogde  aandacht  evenwel  niet  worden  gegeven, 
vooral  niet  bij  hoge  werklast.  Dit  leidde  tot  vergroting  van  de  volgfbut.  Aanbevolen  wordt 
in  een  vervoigstudie  na  te  gaan  hoe  prestatie  en  orientatie  van  de  operator  kunnen  worden 
verbeterd  bij  lage  bijwerkfrequenties,  bijvoorbeeld  door  integratie  in  het  sensorbeeld  van 
synthetische  informatie  over  MUAV  status  en  orientatie  (“synthetic  image  augmentation”). 
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1  INTRODUCTION 
1.1  Background 

Maritime  Unmanned  Air  Vehicles  (MUAVs)  are  airframes  with  a  payload  that  are  launched 
and  propelled  through  the  air  without  an  operator  on  board.  Instead,  these  vehicles  are 
remotely  controlled  by  an  operator  who  is  situated  in  a  base  control  station.  The  Royal 
Netherlands  Navy  (RNLN),  as  well  as  other  participating  countries  in  NATO  Naval 
Armament  Group  (PG/35),  are  interested  in  the  development  of  these  air  vehicles,  in 
particular  for  anti-surface-warfare  purposes  (NATO  staff  requirement,  1992).  This  includes 
surfece  reconnaissance,  surveillance  and  target  acquisition  (RSTA),  as  well  as  battle  damage 
assessment  (BDA).  As  MUAVs  are  not  strictly  autonomous,  operators  are  closely  involved, 
guiding  the  vehicles  towards  their  targets  while  interpreting  threat,  tactical,  and  sensor 
information.  Especially  when  tracking  targets,  the  operator  has  to  ensure  that  he  is  accu¬ 
rately  following  the  target  while  envisioning  an  environment  which  is  remotely  obtained.  As 
the  sensor  image  has  a  limited  resolution  and  field  of  view  (FOV),  and  no  vestibular 
feedback  of  the  air  vehicle  behaviour  is  obtained,  it  is  obvious  that  operator  behaviour  is 
crucial  for  effective  MUAV  operation.  Therefore,  the  TNO  Human  Factors  Research 
Institute  was  asked  to  investigate  the  human-machine  parameters  of  the  MUAV  user 
interface  that  affect  operator  performance. 

In  a  first  approach,  a  literature  review  (Eisen  &  Passenier,  1991a)  was  performed,  based  on 
NATO  launch,  mission  and  recovery  requirements  (LaMonica,  1988;  NATO,  1990,  1992; 
Schwartz  et  al.,  1992).  It  was  concluded  that  the  human-machine  interface  should  be 
optimally  designed  for  supervisory  control  (target  acquisition  phase)  and  tracking  for 
RSTA/BDA.  NATO  staff  targets  (NATO,  1990)  indicate  that  MUAVs  should  only  be  used 
under  certain  predetermined  conditions.  For  instance,  a  certain  predetermined  observation 
distance  to  the  target  should  be  maintained  in  order  to  avoid  early  detection  of  the  MUAV. 
Since  target  loss  must  be  limited  to  a  minimum,  it  is  important  to  select  the  proper  technol¬ 
ogy  for  the  MUAV  and  carefully  design  the  MUAV  user  interface.  Eisen  and  Passenier 
(1991b)  also  concluded  that  MUAV  operators  are  involved  in  several  functions  during 
MUAV  missions.  Guidelines  for  information  presentation  were  specified,  such  as  sensor, 
tactical,  and  strategic  information. 

In  a  next  phase,  an  exploratory  experiment  was  conducted  (Van  Breda  &  Passenier,  1993)  in 
which  an  operator  had  to  control  a  MUAV  and  maintain  a  predetermined  constant  observa¬ 
tion  distance  relative  to  a  fast  moving  target.  Doing  so,  the  operator  was  confronted  with 
multiple  degrees  of  control  such  as  altitude,  horizontal  position  and  yaw  of  the  MUAV  as 
well  as  pitch,  yaw  and  FOV  of  the  sensor  system,  each  of  these  affecting  the  size  and 
position  of  the  sensor  footprint.  Results  of  the  experiment  indicated  that  accurate  tracking  is 
only  possible  when  a  control  input  device  is  used  based  on  a  coupled  control  algorithm. 
Only  than,  the  4  Hz  sensor  image  update  frequency,  as  recommended  by  Vitro  (Schwartz  et 
al.,  1992),  will  lead  to  an  acceptable  tracking  performance  by  the  operator.  Update  frequen¬ 
cies  less  than  4  Hz  will  cause  target  loss. 

The  study  only  investigated  tracking  performance  with  single  MUAV  operation.  However, 
the  RNLN  considers  effective  over-the-horizon  MUAV  systems  important  for  heli  operation 


6 


support  (Fortgens,  1994),  in  particular  configured  in  so  called  multi-MUAV  clusters.  Multi- 
MUAV  missions  are  partly  pre-programmed,  partly  manually  controlled.  For  example;  one 
MUAV  for  tracking  targets  or  for  RSTA/BDA,  two  MUAVs  as  relay  stations  (the  first  is 
moving  to  the  target  area  to  take  over  the  tracking  job  as  soon  as  the  actual  tracking  MUAV 
has  to  return  to  base  for  refuelling;  the  second  is  moving  from  the  target  area  back  to  base 
to  get  refuelled),  and  one  MUAV  is  being  refuelled  at  the  base.  The  operator  is  involved  in 
a  supervisory  control  task:  supervision  of  tracking,  tactical,  strategic  and  MUAV  status 
information,  and  simultaneously  tracking  targets  manually.  Since  operator  workload  may 
play  an  important  role  under  these  operational  circumstances,  an  experiment  was  conducted 
to  determine  the  extent  to  which  tracking  performance  is  affected  by  the  supervisory  tasks. 
This  report  describes  this  experiment  and  the  results. 

During  multi-MUAV  missions,  only  the  tracking  MUAV  is  under  direct  control  by  an 
operator,  whereas  the  other  air  vehicles  are  more  or  less  autonomously  flying  from  waypoint 
to  waypoint,  or  waiting  and  hovering  at  predetermined  positions,  until  action  is  needed.  A 
global  function  description  of  a  multi-MUAV  mission  contains  the  following  main  functions: 

-  Track  targets 

Once  the  targeting  MUAV  is  at  a  predetermined  observation  (stand-off)  distance,  the 
operator  uses  the  control  input  devices  and  visual  feedback  to  minimize  the  deviation 
between  the  crosshair  of  the  sensor  image  and  the  moving  target.  It  should  be  noticed  that 
a  MUAV  is  a  relatively  fast  responding  system  which  has  severe  restrictions  in  minimum 
observation  distance  (forward  distance  to  the  target  to  be  maintained)  and  field  of  view 
(FOV)  of  the  sensor.  Information  concerning  the  target’s  future  position  may  be  derived 
from  the  perceived  target  position  and  orientation  (perceptual  anticipation)  and  from 
knowledge  of  the  MUAV  and  target  movement  (cognitive  anticipation).  Since  the  primary 
mission  of  the  MUAV  system  in  the  present  study  is  probably  RSTA,  the  a-priori 
information  on  target  characteristics  is  minimal.  Therefore,  with  regard  to  target 
tracking,  cognitive  anticipation  is  hardly  supported. 

-  Supervise 

The  MUAV  operator  has  to  supervise  a  multiple  MUAV  system,  monitoring  tactical 
mission  information  and  control /status  information  to  judge  the  state  of  the  air  vehicles. 
Therefore,  the  mission  scenario  should  be  considered  as  a  procedure,  supervised  by  the 
operator.  He  verifies  the  different  system  parameters  and  take  the  necessary  corrective 
actions. 

-  Communicate 

The  MUAV  operator  communicates  with  the  Command.  It  is  suggested  that  the  operator 
has  an  open  connection  through  the  communication  network  using  a  headset,  scanning 
messages  that  are  assigned  to  him.  It  is  essential  that  the  operator  recognizes  these 
messages  correctly  and  acts  accordingly. 

The  RNLN  is  interested  to  know  whether  a  single  MUAV  operator  is  able  to  perform 
multiple  tasks  in  order  to  fulfil  the  functions  described  above.  It  seems  logical  to  think  that 
tasks,  being  performed  simultaneously,  affect  each  other  (e.g.  calculating  and  discussing). 
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But  it  is  also  known  that  task  combinations  exist  that  can  very  well  be  performed  in  parallel 
(e.g.  steering  a  car  and  discussing  with  someone,  or  listening  to  the  radio).  Therefore, 
multiple  task  performance  should  be  investigated  before  a  new  system  is  developed.  A  model 
that  can  be  used  to  evaluate  human  interface  principles  and  to  predict  workload,  is  the 
multiple  resource  model  developed  by  Wickens  (1984).  This  model  is  specific  for  multiple 
(continuous)  task  performance  and  distinguishes  three  resources  (see  also  Veltman  & 
Gaillard,  1991): 

-  modalities;  channels  through  which  information  is  obtained  (e.g.  visual,  verbal),  or  the 
way  response  is  generated  (e.g.  verbal,  manual); 

-  stages  of  information  processing;  perceive,  central  processing,  respond; 

-  coding;  the  way  information  is  being  processed.  This  can  be  spatial  (e.g.  orientational 
tasks)  or  verbal  (e.g.  retaining  names). 

The  model  predicts  that,  when  (independent)  parallel  tasks  use  the  same  resource  (interfere), 
the  task  performance  on  each  individual  task  will  decrease.  This  will  not  happen  when 
parallel  tasks  use  different  resources.  For  instance,  verbal  tasks  can  easily  be  combined  with 
visual  tasks;  a  task  in  an  early  phase  of  information  processing  (e.g.  perceive,  pattern 
recognition)  can  easily  be  combined  with  a  task  that  generates  a  response  (e.g.  steering);  a 
task  that  processes  spatial  information  (e.g.  interpreting  radar  information)  can  easily 
combined  with  a  verbal  task  (e.g.  communicate). 


1.2  The  experiment 

In  order  to  determine  the  effect  of  verbal /cognitive  workload  on  tracking  performance,  an 
experiment  was  conducted.  Subjects  had  to  approach  a  moving  target  in  an  open  sea  scenario 
and  follow  this  target  at  a  predetermined  stand-off  distance.  In  the  experiment,  a  coupled 
control  input  device  (Van  Breda,  1993)  was  used,  in  which  one  single  Joystick  controlled 
footprint  and  air  vehicle  (Eisen  &  Passenier,  1991b).  The  control  signal  was  interpreted  by  a 
control  algorithm  for  maintaining  a  fixed  stand-off  distance  (2000  m).  If  the  target  distance 
varied,  the  sensor  footprint  could  be  manually  controlled  in  forward  or  backward  direction 
by  pushing  or  pulling  the  Joystick.  The  corresponding  air  vehicle  motions  were  calculated 
and  executed.  By  moving  the  Joystick  sideward,  the  horizontal  viewing  angle  was  controlled 
by  the  yaw  rate  of  the  MUAV  air  vehicle. 

To  vary  workload,  a  verbal/cognitive  task  had  to  be  performed  in  addition  to  this  visual- 
manual-spatial  tracking  task.  This  so  called  continuous  memory  task  (CMT)  (Boer,  1992; 
Veltman  &  Gaillard,  1993)  was  presented  to  the  subjects  by  means  of  a  verbal  display 
(headset)  and  was  characterized  by  detecting  certain  target  letters  and  mentally  counting  the 
frequency  of  their  appearance.  The  task  difficulty  was  varied  by  the  number  of  target  letters. 
The  subjects  were  asked  to  perform  the  tracking  task  as  accurately  as  possible,  but  to  give 
the  CMT  the  highest  priority. 

It  should  be  noticed  that  the  CMT  only  globally  reflects  the  future  MUAV  operator  task 
aspects  while  fulfilling  the  functions  as  described  in  §  1.1.  It  is  obvious  that  specific  visual 
or  manual  tasks  (for  instance  emergency  diagnosis  and  control  tasks  during  erroneous 
MUAV  functioning)  could  affect  the  operator  task  performance  differently.  This  experiment 
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is  considered  as  a  first  test,  so  a  normative  approach  was  taken:  the  operator  will  not  make 
procedural  errors  and  no  vehicle  malfunctioning  will  occur. 

According  to  a  former  experiment  (Van  Breda  &  Passenier,  1993),  the  sensor  image  update 
frequenqr  is  one  of  the  crucial  fectors  to  affect  operator  tracking  performance.  Therefore, 
both  image  update  and  CMT  workload  were  varied  in  the  experiment.  With  respect  to  the 
fector  ‘image  update  frequency’,  the  sensor  video  image  was  supposed  to  be  obtained 
through  a  digital  datalink,  resulting  in  image  update  frequencies  varying  from  1  to  10  Hz. 
For  the  factor  ‘cognitive  workload’,  the  difficulty  of  the  CMT  was  varied  at  three  levels: 
‘low’,  ‘moderate’  and  ‘high’. 

In  summary,  the  MUAV  operator  has  to  minimize  deviations  between  the  crosshair  of  the 
sensor  image  and  the  moving  target.  Based  on  the  former  experiment,  it  can  be  expected  that 
low  sensor  image  update  frequencies  will  deteriorate  perceptual  anticipation,  causing 
relatively  large  viewing  errors,  and  therefore  decrease  the  operator  tracking  performance. 

Wickens’  multiple  resource  model  predicts  that  the  (visual/manual)  spatial  tracking  task  can 
well  be  combined  with  a  verbal/cognitive  task.  Therefore,  with  regard  to  the  verbal/ 
cognitive  workload  introduced  by  the  CMT,  it  can  be  expected  that  high  workload  will 
hardly  affect  the  tracking  task  performance. 


2  METHOD 
2.1  Subjects 

Ten  male  subjects  participated  in  the  experiment.  These  subjects  were  starting  College 
students.  Their  age  varied  from  21  to  39  years  (average  25.6  years). 


2.2  Task 

Each  subject  was  sitting  at  a  desk,  providing  a  display  and  joystick  for  tracking  targets,  and 
a  display  and  headset  for  the  additional  CMT.  During  each  trial,  the  subjects  initially  flew  a 
MUAV  at  5000  m  distance  from  a  fast  moving  target  ship.  This  target  ship  had  to  be 
approached  until  2000  m  stand-off  distance.  During  this  approach,  the  sensor  remained 
zoomed  out  (46.7°  FOV).  Once  the  desired  stand-off  distance  was  obtained,  the  subject  had 
to  zoom  in  on  the  target  ship  (3°  FOV)  and  track  the  target  ship,  which  means  to  keep  the 
crosshairs  of  the  sensor  image  as  accurately  as  possible  on  a  special  marker  at  the  stem  of 
the  target  ship  (tracking  phase).  However,  this  target  ship  performed  unexpected  course 
changes.  In  case  of  target  loss,  it  was  allowed  to  zoom  out  and  retrace  the  target  ship. 

At  a  certain  moment,  an  additional  task  had  to  be  performed.  If  so,  a  set  of  (two  or  four) 
target  letters  would  appear  on  a  screen  and  a  button  for  acceptance  had  to  be  pushed,  to  start 
the  CMT.  A  series  of  40  letters  were  presented  in  the  subject’s  headset:  one  letter  every 
1.5  s.  The  subject  had  to  press  a  ‘recognition’  button  once,  each  time  he  recognized  one  of 
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the  target  letters.  Moreover,  he  had  to  push  the  ‘recognition’  button  twice,  each  time  any 
target  letter  was  recognised  for  the  third  time.  In  case  a  double  response  was  expected,  the 
subject  heard  ‘Correct’  when  giving  a  double  response;  the  subject  heard  ‘Wrong’  once  an 
omission  was  made  or  when  the  subject  responded  at  the  wrong  moment.  After  the  response 
(either  ‘Correct’  or  ‘Wrong’)  the  tally  for  the  letter  that  was  presented,  was  reset  to  zero.  A 
CMT  with  two  target  letters  was  considered  to  be  of  ‘moderate  workload’,  a  CMT  with  four 
target  letters  of  ‘high  workload’  (see  also  Aasman,  Mulder  &  Mulder,  1987;  Veltman, 
1991). 

The  subject  was  instructed  to  track  the  target  ship  as  accurate  as  possible,  using  the  coupled 
control  system.  Although,  the  additional  verbal/cognitive  task  had  to  be  considered  as  the 
highest  priority  task. 

Each  trial  was  finished  after  four  minutes  of  tracking.  Than,  the  next  trial  would  start. 


2.3  Experimental  design 

The  values  of  the  independent  variables  were  varied  as  follows: 

1  Image  update  frequency  (3  levels):  1  Hz,  4  Hz  and  10  Hz. 

2  Verbal/cognitive  workload  (3  levels):  low,  moderate,  high. 

These  variables  were  independently  combined  within  subjects  in  a  balanced  order.  This 
made  up  to  a  total  of  9  conditions. 


2.4  Instrumentation 

The  experiment  was  carried  out  on  the  MUAV  simulator  developed  at  the  TNO  Human 
Factors  Research  Institute.  This  simulator  was  based  on  a  Silicon  Graphics  Iris  4D  work¬ 
station  for  calculation  of  the  position  of  the  footprint  and  for  generation  and  presentation  of 
the  tactical  and  sensor  information.  The  subject  was  seated  in  a  cubicle  in  which  a  monitor 
and  control  devices  were  installed.  Control  actions  of  the  subject  were  fed  into  the  Iris  in 
which  a  mathematical  model  of  the  target  ship  behaviour  and  of  the  MUAV  including  sensor 
characteristics  were  implemented.  A  right-hand  Joystick  was  used  for  combined  control  of 
sensor  and  MUAV  motion  in  the  so-called  coupled  control  mode  (Van  Breda  &  Passenier, 
1993). 

For  generation  and  presentation  of  the  CMT  task,  a  Taskomat  was  used  (Spoelstra,  1993), 
consisting  of  a  IBM  PC  with  timer  response  and  speech  interlace  for  stimulus  generation  and 
verbal  presentation. 

Detailed  information  on  simulator  characteristics,  MUAV  and  target  behaviour  are  presented 
in  the  Appendix. 
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2.5  Procedure 

The  subjects  came  in  pairs  during  one  day.  First,  they  were  introduced  to  the  MUAV 
simulator  by  the  instructor,  who  explained  the  control  input  devices  and  the  presented 
information  on  the  screen.  Then,  a  training  session  took  place  in  which  three  tracking  test- 
trials  (update  frequency  of  10,  4,  1  Hz  respectively)  had  to  be  performed.  Finally,  the 
subjects  were  trained  to  perform  the  CMT  (moderate,  high  workload  respectively),  sepa¬ 
rately  from,  and  simultaneously  with  the  tracking  task.  The  task  performance  was  verified 
by  the  instructor. 

During  the  actual  experiment,  each  subject  had  to  perform  6  blocks  of  3  trials  each.  When 
one  subject  was  performing,  the  other  was  waiting  and  took  over  after  each  block.  The 
instructor  closely  monitored  the  subject’s  task  perfiarmance.  In  particular  the  CMT  perform¬ 
ance  had  to  be  monitored  and  kept  at  a  maximum  level. 


2.6  Dependent  variables  and  analysis 

The  sample  rate  for  collecting  data  of  the  state  variables  was  4  Hz.  For  the  analysis  the 
following  dependent  variables  were  calculated,  representing  the  operator  performance: 

1  target  coverage:  percentage  of  time  during  which  the  target  ship’s  stern  marking  was 
visible  on  the  screen; 

2  viewing  error:  the  total  angular  difference  between  the  crosshair  of  the  sensor  image  and 
the  marking  on  the  target  ship,  expressed  in  degrees. 

3  CMT  score:  percentage  correct  response  of  target-letter  hits  and  of  count  3  target-letter 
hits. 

The  analysis  concerned  the  root  mean  square  error  (RMS  error)  of  these  variables,  starting 
at  the  moment  that  the  MUAV  was  within  3000  m  observation  distance.  On  these  data,  an 
analysis  of  variance  (ANOVA;  Winer  et  al.,  1991)  was  performed. 


3  RESULTS 
3.1  Target  coverage 

Fig.  1  shows  the  target  coverage,  for  the  three  image  update  and  workload  conditions, 
averaged  over  the  subjects. 

As  can  be  seen  in  Fig.  1,  the  target  coverage  was  decreasing  with  lower  image  update 
frequencies.  This  was  nearly  100%  at  10  Hz,  and  95%  or  more  at  4  Hz,  but  only  about 
60%  at  1  Hz.  According  to  a  3  (Update  frequency)  x  3  (Workload)  ANOVA  this  effect  was 
significant  [F(2, 18)  =  15.09;  p<<0,01].  However,  the  ANOVA  did  not  show  any  effect  of 
workload  (CMT).  A  Tukey  post-hoc  test  indicated  that  the  1  Hz  condition  significantly 
differed  from  the  4  and  10  Hz  condition  (for  both  p<  <0.01). 
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image  update  frequency  (Hz) 


Fig.  1  Mean  target  coverage  as  a  function  of  the  sensor  image  update 
frequency  and  operator  workload,  averaged  over  the  subjects. 


3.2  Viewing  error 

In  Fig.  2  the  sensor  viewing  error  is  presented,  for  three  image  update  and  workload 
conditions,  averaged  over  the  subjects. 

It  can  be  seen  that  the  viewing  error  was  nearly  negligible  at  image  update  frequencies  of  4 
Hz  or  more,  and  was  increasing  significantly  at  low  update  frequencies.  According  to  a  3 
(Update  frequency)  x  3  (Workload)  ANOVA  this  effect  was  just  significant  [F(2,18)  = 
3.36;p=0.05]. 

In  the  figure,  the  screen  limits  (e.g.  the  distance  from  screen-centre  to  screen-edges)  are 
shown,  indicating  the  visibility  criterion  of  the  tracking  task.  It  can  be  seen  that  the  viewing 
error  only  exceeded  the  screen  limits  at  the  lowest  update  frequency  condition.  This  was  also 
the  only  condition  in  which  any  effect  of  workload  was  found.  The  ANOVA  did  not  indicate 
any  significance  of  this  effect. 
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image  update  frequency  (Hz) 


Fig.  2  Mean  sensor  viewing  error  as  a  function  of  the  sensor  image  update 
frequency  and  operator  workload,  averaged  over  the  subjects.  The  horizontal 
dashed  area  indicates  the  screen  limits  (distance  from  screen-centre  to  screen- 
edges). 


3.3  CMT  score 

In  Fig.  3  the  percentage  correct  response  of  target-letter  hits  and  count  3  target-letter  hits  are 
presented,  for  three  image  update  and  workload  conditions,  averaged  over  the  subjects. 

The  results  clearly  show  that  the  percentage  correct  response  of  target-letter  hits  was  about 
100%  for  all  conditions,  no  matter  the  image  update  or  workload  condition.  A  3  (Update 
frequency)  x  3  (Workload)  ANOVA  did  not  indicate  any  significant  effect  or  interaction. 

The  figure  also  shows  that  the  average  percentage  count  3  target-letter  hits  was  about  90% 
and  had  a  tendency  to  decrease  with  low  update  frequencies.  However,  the  ANOVA  did  not 
confirm  this  effect.  The  effect  of  workload  is  clear,  the  performance  at  high  workload  was 
about  10%  less  compared  to  the  performance  at  moderate  workload.  This  effect  was 
significant  [F(l,9)=  9.43;  p<0.05]. 


13 


lOOh 


O 

w 

c 

o 

Q. 

w 

o 


o 

(D 

k. 

o 

o 

o 

Ui 

CO 

4-> 

c 

0) 

o 

u. 

CD 

a 


80 


60 


40 


20 


0 


- 1 - 

- 1 - , - 

- a  — 

w  - -  w 

„.,o . ° 

o’ 

. . 

...• . 

- 

- 

•  high  workload 

o  moderate  workloacT 

hit 

1 

. hit  count  3 

1 .  1 

1 

4  1 0 

image  update  frequency  (Hz) 


Fig.  3  Mean  percentage  correct  response  of  target-letter  hits  and  count  3 
target-letter  hits,  as  a  function  of  the  sensor  image  update  frequency  and 
workload,  averaged  over  the  subjects. 


4  DISCUSSION 

As  was  expected,  the  experimental  results  showed  a  decrease  of  MUAV  tracking  perfor¬ 
mance  with  lower  update  frequencies  of  the  sensor  image.  With  10  or  4  Hz,  the  mean 
coverage  of  the  target  remained  at  nearly  100%  of  the  tracking  time,  but  at  1  Hz  this  was 
only  about  60%.  This  was  caused  by  an  increase  of  the  viewing  error,  exceeding  the  field  of 
view  of  the  camera. 

As  was  predicted  by  Wickens’  multiple  resource  model,  no  significant  decline  in  target 
coverage  was  found  when  the  subjects  performed  the  tracking  task  under  moderate  and  high 
verbal/cognitive  workload  conditions.  Only  with  1  Hz  update  frequency  the  mean  viewing 
error  increased.  Hence,  Wickens’  predictions  on  task  performance  were  not  completely  valid 
with  regard  to  the  most  difficult  tracking  condition.  Tracking  with  low  update  frequencies 
probably  required  more  central  (mental)  processing  concerning  the  target’s  behaviour.  This 
increasing  demand  of  attention  could  not  be  rewarded,  in  particular  not  during  high 
workload  conditions,  since  the  subjects  were  instructed  to  give  primary  attention  to  the 
verbal/cognitive  task.  This  resulted  in  an  increase  of  the  mean  viewing  (tracking)  error. 
Verification  of  the  subjects’  verbal/cognitive  task  performance  pointed  out  that  this  task  was 
properly  executed;  the  target-letter  hits,  as  well  as  the  count  3  target-letter  hits  remained 
unchanged  over  the  workload  conditions. 
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5  CONCLUSIONS  AND  SUGGESTIONS 

The  RNLN  considers  to  perform  a  MUAV  mission  by  multiple  MUAV  systems,  in  which 
multiple  functions  (track  a  target,  supervise  tactical  information  and  multiple  air  vehicle 
status,  communicate)  have  to  be  fulfilled  simultaneously.  In  order  to  investigate  the  effect  of 
workload  on  tracking  performance,  an  experiment  was  conducted  in  which  operators  had  to 
perform  a  MUAV  supervisory  control  task.  During  this  experiment,  subjects  had  to  track 
fast  moving  targets  in  an  open  sea  situation  with  different  image  update  conditions.  In 
addition  they  had  to  perform  a  verbal /cognitive  task  under  different  workload  conditions. 
Results  of  the  experiment  show  that  moderate  and  even  high  workload  conditions  did  not 
affect  the  mean  target  coverage  of  the  sensor  image.  This  was  about  100%  at  image  update 
frequencies  of  4  Hz  and  more,  decreasing  to  about  60%  at  1  Hz  update  frequency.  This  may 
be  explained  by  the  fact  that,  in  the  latter  case,  the  tracking  task  became  more  discontinu¬ 
ous,  which  demanded  more  attention  for  anticipation.  Since  the  subjects  were  instructed  to 
give  primary  attention  to  the  verbal/cognitive  task,  this  extra  attention  was  not  fully 
available,  in  particular  not  during  high  workload  conditions.  This  resulted  in  an  increase  of 
the  mean  tracking  error. 

The  fact  that  a  significant  tracking  performance  degradation  was  found  at  low  image  update 
frequencies,  indicates  that  image  update  frequency  is  one  of  the  crucial  factors  affecting 
operator  performance.  Many  attempts  are  currently  made  to  improve  the  MUAV  downlink 
transmission  bandwidth  in  order  to  increase  information  flow.  However,  high  costs  and 
technological  limitations  hamper  the  progress  in  this  field  (Schwartz  et  al.,  1992;  NATO, 
1993;  Rochus  &  Garcia,  1995;  Van  Breda,  1995).  Since  the  decrease  in  operator  perform¬ 
ance  is  caused  by  lack  of  anticipation  and  orientation  (Van  Breda  &  Passenier,  1993),  it 
should  be  investigated  whether  provisions  for  enhanced  visual  information  may  be  a  more 
efficient  way  to  improve  operator  performance.  This  should  not  only  concern  improvement 
of  the  sensor  image  characteristics,  but  also  improvement  of  the  operator’s  awareness  by 
depicting  additional  graphics  information  onto  the  sensor  image  (“synthetic  image  augmenta¬ 
tion”).  For  example,  artificial  vehicle  references,  or  synthetic  perspective  graphics  of  a 
“virtual  landscape”  as  seen  from  the  actual  MUAV  position,  displayed  at  a  high  update  rate 
(i.e.  60  Hz),  may  improve  the  operator’s  anticipation  and  orientation  possibilities,  and 
thereby  enhance  the  tracking  performance. 
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APPENDIX:  Simulation  Parameters 


Target  area 

•  Surrounded  by  sea:  a  horizontal  plane  of  infinite  size  and  dark  grey  colour. 

•  Sky:  background  of  light  bluish-grey. 

•  Small  area;  an  area  of  uncertainty  (AOU)  of  100  nm^  provided  by  cue;  circular  in  shape. 

•  Large  area:  1/3  of  a  sector,  uncued.  (A  sector  is  defined  as  a  40°  region,  between  20  and 
100  nm  distance  from  ship;  thus,  1/3  of  a  sector  is  approx.  1100  nm^.) 


Target 


class 

frigate 

length 

100 

m 

width 

15 

m 

height 

4 

m 

maximum  speed 

40 

kts 

turning  radius 

400 

m 

Colour 

•  Dark  grey  hull,  lighter  grey  deck. 

•  Colours  approach  that  of  sea,  linearly  in  RGB  (voltages  of  red,  green,  and  blue  electron 
guns  of  monitor),  as  distance  from  UAV  increases  (to  simulate  atmospheric  effects  on 
image).  At  10  km  distance,  ship  and  sea  colours  are  identical,  simulating  zero  visibility. 

mke 

•  Light  grey  at  ship;  approaches  colour  of  water  at  tail;  visibility  reduces  with  distance 
from  UAV  (in  same  manner  as  ship  does). 

•  Width  is  roughly  equal  to  width  of  ship. 

•  Length  is  a  function  of  ship  speed  (it  lasts  for  20  s;  maximum  is  417  m). 

Manoeuvring 

The  ship  always  moves  at  its  maximum  speed.  Every  10  s,  the  ship  has  a  50%  probability  of 
turning  in  a  random  direction.  Once  a  turn  is  initiated,  the  duration  of  the  turn  is  randomly 
chosen  between  0  s  and  10  s.  The  intention  is  to  simulate  a  range  between  undisturbed 
straight  navigation  when  the  UAV  is  distant,  and  avoidance  manoeuvres  when  the  UAV  is 
near.  The  latter  assumes  a  worst-case  scenario:  the  target  pilot  knows  he  is  under  scrutiny, 
and  wishes  to  make  his  ship  difficult  to  track. 
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MUAV  motion 

Horizontal  translations 

•  maximum  translational  rate  of  +  120  kts 

•  maximum  yaw  rate  of  +  3  °/s 

•  acceleration  control  by  longitudinal  axis  of  position  joystick  or  arrow  keys  (separate 
control  only) 

•  rate  of  yaw  control  by  lateral  axis  of  position  joystick  or  arrow  keys 

•  first-order  control  system  with  3-s  time  constant. 

'Wrtical  translations 

•  constant  altitude  6561  ft. 


Sensor/Gimbal 
Azimuth  (Yaw) 

•  maximum  rate  of  +  30  °/s 

•  360°  range 

•  rate  controlled  by  lateral  axis  of  position  joystick  or  arrow  keys  (separate  control  only) 

•  gain  is  proportional  to  FOV,  allowing  a  constant  sensitivity  to  input  across  all  levels  of 
zoom 

•  gain  is  proportional  to  distance  to  footprint  centre,  allowing  a  constant  sensitivity  to  input 
across  all  distances. 

Elevation  (Pitch) 

•  maximum  rate  of  +  20  °/s 

•  range  of  20°  to  -120° 

•  rate  controlled  by  longitudinal  axis  of  position  joystick  or  arrow  keys 

•  gain  is  proportional  to  FOV,  allowing  a  constant  sensitivity  to  input  across  all  levels  of 
zoom 

•  gain  is  proportional  to  distance  to  footprint  centre,  allowing  a  constant  sensitivity  to  input 
across  all  distances. 

Field  of  view 

•  maximum  FOV  of  46.7° 

•  minimum  FOV  of  3.0° 

•  FOV  increase/decrease  controlled  by  function  keys. 

Coupled  control 

Single  input  device  controls  UAV  yaw  laterally  and  footprint  speed  longitudinally. 
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